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ABSTRACT 



This document contains three papers from a symposium on new 
perspectives in education that was conducted as part of a conference on human 
resource development (HRD) . "A Systems Model for Evaluating Learning and 
Performance" (Hallie Preskill, Darlene Russ-Eft) describes a model for HRD 
developers to use in evaluating learning and performance that takes into 
account not just the processes of evaluation but also the internal and 
external variables that affect evaluation design, implementation, and 
utilization. "Economic Analysis of Human Resource Development: Update on the 
Theory and Practice" (Richard A. Swanson) reviews economic research related 
to the contribution HRD makes to the success of organizations and emphasizes 
the importance of assessing HRD functions in terms of their overall 
contribution to the organization. According to "Evaluating HRD Research Using 
a Feminist Research Framework" (Laura L. Bierema, Maria Cseh) , which reports 
on an analysis of more than 400 Academy of Human Resource Development papers 
from 1997-1999, few studies recognize gender as a category of analysis and 
studies concerned with the following topics are nearly absent from the HRD 
literature: women's/diverse people’s experience, asymmetrical power 
arrangements, problems of racism and sexism, and advocacy of social justice 
and change. All three papers contain substantial bibliographies. (MN) 
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A Systems Model for Evaluating Learning and Performance 



Hallie Preskill 
University of New Mexico 

Darlene Russ-Eft 
AchieveGlobal 



In this paper we describe a model for evaluating learning and performance that takes into account 
not only the processes of evaluation , but also the internal and external variables that affect the 
design and implementation of an evaluation, as well as the use of its findings. The proposed 
model is grounded in the belief that evaluation can facilitate and support individual, team and 
organizational learning, and that HRD professionals should assume the role of internal 
evaluators. 
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With increasing frequency, learning and performance professionals are being asked to evaluate the effectiveness and 
impact of their efforts (Bassi, Benson, & Cheney, 1996; Bassi & Lewis, 1999; Brown & Seidner, 1998). As Phillips 
(1998) notes, “Although organizations have focused much attention on evaluation in the past 40 years, only recently 
have organizations taken a systematic and comprehensive approach to evaluating training and development, human 
resource development, and performance improvement initiatives” (p. 1). The urgency to evaluate has emerged from 
both internal and external customers who are asking for evidence of programs’ effectiveness and contributions. The 
demand for, and interest in, systematic and useful evaluation results are occurring not only in for-profit 
organizations, but in non-profit and local, state, and federal agencies as well. 

While we believe this increased attention to evaluation represents an extremely important development, it has 
presented HRD professionals with a significant challenge. First, few trainers or other learning and performance 
practitioners have any formal education or training in the philosophies, theories or technical skills of evaluation. 
Most HRD professionals know what they know about evaluation from either taking brief workshops on the topic, or 
through their experiences with evaluation on the job. Few are aware that evaluation constitutes a profession with its 
own history, theories, and standards. Nor do they know that professional evaluation organizations exist all across 
the world (e.g., American Evaluation Association, Canadian Evaluation Society, Australasian Evaluation 
Association, European Evaluation Society). The second dilemma trainers face is that the models on which they Ve 
. relied to conduct evaluation are far too simplistic and vague to guide them in the kinds of evaluation work now 
being required (Hilbert, Preskill & Russ-Eft, 1997). 

For example, Donald Kirkpatrick^ four-level evaluation model (1959 a, b; 1960 a, b; 1994), developed 
over 40 years ago continues to offer HRD practitioners and researchers little more than a taxonomy of evaluation 
categories. Ask almost any trainer and they know that Level 1 means measuring a participant^ reactions to the 
learning event; Level 2 is associated with measuring how much participants’ learned; Level 3 relates to determining 
the extent to which participants’ on-the-job behavior has changed as a result of the training; and Level 4, the holy 
grail of all training evaluation, concerns measuring the results of the learning intervention on the organization, 
otherwise referred to as cost-benefit analysis or measuring the retum-on-investment. 

While others have attempted to expand on Kirkpatrick^ model over the last four decades ( Brinkerhoff, 
1989; Hamblin, 1974; Holton, 1996; Phillips, 1995), none has provided a comprehensive, systems-approach, 
practitioner-oriented evaluation model. If HRD professionals are to demonstrate the effects of their efforts, then 
they must be provided with the necessary knowledge and tools to competently evaluate their programs, products, 
processes, services, and systems. In essence, they must become internal evaluators. 

What follows is an evaluation model that we hope will take HRD professionals to the next level of 
conceptualizing and practicing evaluation within their organizations. In the following pages we outline the 
questions that guided the modelfc development, describe its theoretical underpinnings, and explain each of the 
modelfc components. We conclude with our ideas on how the evaluation model may contribute to new evaluation 
practice and research. 
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Theoretical and Research Questions: 



The models development grew out of the following questions: 

1. What are the necessary components of a comprehensive systems-approach to evaluating learning and 
performance within organizations? 

2. What system- wide variables affect the evaluation of learning and performance efforts within an organization? 

3. How should evaluation be practiced within an organizational context? 



Theoretical Framework 

The evaluation model we propose is based on the philosophies, theories and practices in the fields of: 1) 
evaluation, 2) organizational learning, and 3) systems theory. While several definitions of evaluation have been 
offered in the evaluation literature (see Patton, 1997; Rossi & Freeman, 1985; Scriven, 1967, 1991), the model 
presented in this paper is based on the following understanding of evaluation: 

Evaluative inquiry is an ongoing process for investigating and understanding critical organization 
issues. It is an approach to learning that is fully integrated with an organization^ work practices, 
and as such, it engenders (a) organization members’ interest and ability in exploring critical issues 
using evaluation logic, (b) organization members’involvement in evaluative processes, and (c) the 
personal and professional growth of individuals within the organization. (Preskill & Torres, 1999, 

p. 1-2) 

Implied in this definition is the notion that evaluation is a systematic process. It should not be conducted as 
an afterthought; rather, it is a planned and purposeful activity. Second, evaluation involves collecting data regarding 
questions or issues about society in general and organizations and programs in particular. Third, evaluation is seen 
as a process for enhancing knowledge and decision-making, whether the decisions are related to improving or 
refining a program, process, product, system, or organization, or for determining whether or not to continue or 
expand a program. In addition, evaluation involves some aspect of judgment about the evaluancte merit, worth or 
value. Finally, the notion of evaluation use is embedded in all evaluation activity. 

In the last decade, many organizations have embraced the concept and practices of organizational learning 
in an effort to respond to constantly changing global economic, technological, and social conditions. Fiol and Lyles 
(1985) define organizational learning as changes in the organization cognition or behavior. This learning 
represents itself in ‘the development of insights, knowledge, associations between past actions, the effectiveness of 
those actions, and future action” (p. 81 1). Thus, we believe that evaluation can be the mechanism for inquiring into 
a problematic situation on the organization^ behalf (Argyris & Schon, 1996), and for helping organization members 
grow and learn as a result of such inquiry. 

Our model has also been strongly influenced by the concepts of systems theory and systems thinking 
(Senge, 1990; Wheatley 1992). Adopting a systems perspective means that we look at an organization as a set of 
interrelated parts and interconnected systems, which are dependent on one another and influenced by each other. As 
Senge (1990) explains, ‘It is a framework for seeing interrelationships rather than things, for seeing patterns of 
change rather than static Snapshots’... it is a discipline for seeing the Structures ’that underlie complex situations, 
and for discerning high from low leverage change” (p. 68-69). A systems approach to organizational learning and 
change acknowledges that an organization is, 

Interacting with its environment and has to adapt to it and permanently change in order to 
survive... The organization is conceptualized as an information processing system, a systems 
which performs certain necessary functions such as the generation of information, as well as the 
diffusion, the storage and the utilization of this information... This systemic approach aims at 
describing the way an organization can learn as a system” (Finger & Brand, 1999, p. 138). 

Therefore, information is a core element of organizations that learn. Emphasizing the role of information in 
organizations, Wheatley (1992) passionately writes, “The fuel of life is new information-novelty-ordered into new 
structures. We need to have information coursing through our systems, disturbing the peace, imbuing everything it 
touches with new life” (p. 105). Since evaluation provides information for decision-making and learning, we 
believe that evaluation can serve a critical role in organizations. 
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A Systems Model of Evaluation 

As depicted in Figure 1, the evaluation models general conception denotes a non-linear understanding of 
evaluation practice, though the evaluation process itself occurs in a somewhat linear manner. Starting with the 
center circle, we outline five critical evaluation phases or processes. Every evaluation starts with the Focusing 
phase. This is where primary stakeholders of the program come together to discuss the background and history of 
the evaluand (that which is being evaluated), to identify other stakeholders who might be interested in using the 
evaluation results, and key questions the evaluation will address. This process is fundamental to ensuring that the 
evaluation attends to issues of concern to a variety of individuals and groups. Throughout our teaching, training and 
consulting experiences, we have found that all too often trainers use the term Evaluation” synonymously with 
Eurvey.” Without first knowing what questions we wish to answer from an evaluation, it is difficult to know which 
methods should be used to collect valid data. The second phase of the evaluation process is determining the 
evaluation^ design , methods of data collection , and means for implementation . There should be discussions about 
which methods (e.g., surveys, tests, interviews, observation, document reviews) would be best suited to answering 
the key evaluation questions. In addition, the team should talk about how to ensure the validity of data, how to 
obtain a high response rate, the timing of data collection, and how to choose a sample population if the entire 
population cannot be involved. The next phase in the evaluation involves analyzing the data , as well as interpreting 
and assigning meaning to the data. This process typically involves statistical analysis of quantitative data and/or 
content analysis of the qualitative data. The fourth phase of the evaluation process is communicating and reporting 
the evaluation findings. Though it appears as a final phase in the process, it should be noted that it is often 
important to communicate the evaluation^ progress and sometimes, preliminary findings, to various stakeholders or 
audiences during the evaluation. HR D professionals who do evaluation should also seek ways to disseminate the 
findings via various communication devices such as memos, posters, and newsletters, in addition to the usual report 
(Torres, Preskill, & Piontek, 1996). The fifth part of the process is making sure to have one or more management 
plans that describe in detail, the logistics of the evaluation. There may be several different plans including a time- 
line, a roles and responsibilities plan, a communicating and reporting plan, a budget, and/or a plan of options in case 
obstacles are encountered during the evaluation^ implementation. 

What we have just described is standard practice for most evaluators, and by itself, does not constitute the 
uniqueness of this model. It is the remaining components that have gone unexplained in the HRD evaluation 
literature. As can be seen in Figure 1, the Evaluation Process sits within an internal organizational context that 
includes three components: 1 ) Evaluator Characteristics , 2) Political Context , and 3) Intended Use of Findings. We 
provide these additional components because of their potential affect on the evaluation^ commissioning, design, 
implementation, and reporting of findings. Evaluation does not occur in isolation; it is affected by several different 
personal and organizational factors. One of these factors is the evaluatorfc characteristics. These characteristics 
include the evaluatorfc: 

• credibility 

• experience with the program 

• previous experience in conducting evaluation 

• knowledge of evaluation theory and methods 

• position within the organization relative to the program being evaluated 

• commitment to evaluation and use of findings 

• understanding of the organization^ culture and politics 

• commitment to ethical behavior - integrity 

• group facilitation skills 

• verbal and written communication skills 

• understanding of program content 

• data analysis skills 

In order to minimize evaluator bias as much as possible in designing, implementing and reporting the results of the 
evaluation, we recommend establishing a task force, or workgroup that agrees to 
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Figure 1. 
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collaborate on the evaluation. This approach creates a kind of checks and balances throughout the evaluation 
process. Plus, it increases the team and organization^ learning potential by their very involvement in the 
evaluation. Assessing these evaluator characteristics before the evaluation begins, can not only help build a better 
evaluation team, but it can help avoid some serious obstacles once the evaluation is underway. 

Evaluation is inherently a political act. Therefore, it is important that HRD professionals clearly 
understand that regardless of the evaluation fc depth and scope, it deals with issues of power, position and resources. 
In 1973, Carol Weiss was one of the first evaluation researchers to publicly recognize the importance of politics and 
values within the evaluation and policy-making process. She identified three ways in which evaluation and politics 
are related: 1) The policies and programs with which evaluation deals are creatures of political decisions; 2) Because 
evaluation is undertaken in order to feed into decision-making, its reports enter the political arena; and 3) Evaluation 
itself has a political stance. By its very nature it makes implicit political statements about such issues as the 
problematic nature of some programs and the unchallengeability of others, the legitimacy of program goals and 
program strategies, the utility of strategies of incremental reform, and even the appropriate role of the social scientist 
in policy and program formation (1987, p. 47-48). Thus, the proposition, Evaluation is always disruptive of the 
prevailing political balance,” reminds us that even in programs that seem ‘honpolitical,” there are political 
implications from the mere act of evaluation, not to mention the findings (Guba & Lincoln, 1981). As a group 
begins to engage in the evaluation process, they should consider the ways in which the evaluation could be political 
and identify ways in which the politics can be managed. 

The third component is Intended Use of Findings. For over 25 years, evaluation researchers have explored 
the topic of evaluation use or utilization. Michael Patton, who has written most extensively on this topic, suggests 
that evaluation without an intention to use the findings should not be conducted at all (1997). We concur. We like 
to think that use is to evaluation what transfer of learning is to training. In other words, for every evaluation that is 
conducted, there should be a plan for the intentional use of findings; otherwise, investing the resources in an 
evaluation may be questionable. Use of findings can take several forms, however. Most common is what is referred 
to as ‘Instrumental use.” With this kind of use, there is a direct application of the evaluation findings. For example, 
if a particular case study exercise in a training workshop consistently receives low ratings, the trainer may decide to 
eliminate the case and replace it with a different experiential activity. This represents a direct or instrumental use of 
the evaluation findings. 

The second type of use is called Eonceptual” or ‘knowledge” use. This type of use cannot be seen 
directly, as the information from the evaluation becomes integrated with what the liser” already knows or believes 
about the evaluand. The evaluation information serves to inform the individual, and contributes to a higher level of 
understanding or cognition about the program being evaluated. The individual might make a decision later on that 
in part was based on the evaluation findings, but she might be hard pressed to say it was based solely on the 
evaluation^ results. 

Finally, evaluation findings might be used in a Symbolic,” ‘political,” or “persuasive” way. For example, 
many programs are required to participate in an accreditation evaluation, which includes a self-evaluation 
component. Members of the organization might not be particularly interested in the evaluation and thus do it, 
symbolically. They may not use the findings to improve their programs, but they can show that ‘they did it.” 
Another common example is when we use the findings of an evaluation to lobby for additional program resources. 
This political act is a justifiable use of evaluation results. Weiss (1998) suggests that, 

Rather than yearning to free evaluation from the pressures of politics, a more fruitful course would 
be to undertake and report evaluation with full awareness of the political environment. A study 
will be more useful and more likely to be used if it addresses the issues that are in contention and 
produces evidence directly relevant to political disputes (p. 316). 

In many evaluations the results may be used in all three ways: 1) instrumentally for improving the program, 2) 
conceptually to inform others about the program^ effects, or 3) symbolically, to increase the credibility of the HRD 
function. It is important for the evaluation team to discuss these potential uses and to identify anywhere the process 
or findings might be misused before the evaluation is implemented. 

Even if the evaluation process is well defined and articulated, and even if the evaluation team has 
considered the evaluatorfc characteristics, the political context, and the intended uses of the evaluation^ findings, 
evaluation still must be implemented within an even larger organizational context. The outer circle (Figure 1) 
describes the necessary organizational infrastructure for supporting evaluation practice (Preskill & Torres, 1999). 
Though rarely discussed in other HRD evaluation models, we believe that for evaluation to contribute to learning 
and decision-making in organizations, there must be at least some presence of these four elements. The first is the 
organization^ leadership. The more leaders support a learning environment, the more likely organization members 
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will support systematic and ongoing evaluation. If organizational leaders suggest that they know it all, or that 
learning from our experiences is unnecessary, then evaluation will be more difficult to implement. On the other 
hand, if leaders model learning, create a spirit of inquiry, and use data to act, then evaluation practice may be more 
successful. 

The second element, the organization^ culture, is fundamental to creating learning from evaluation 
practice. If the culture is one which supports asking questions, open and honest communication, teamwork, risk- 
taking, valuing mistakes, and employees trust each other, evaluation may be welcomed and successful. However, if 
organization members are afraid to give their opinions, dont believe managers will act on the results, or the results 
will be used to punish individuals or groups, then the evaluation^ results will be less useful in helping the 
organization make effective decisions. 

The third element, organization communications is closely related to the intended use of evaluation 
findings . The more systems and channels an organization has to communicate and report the progress and findings 
of an evaluation, the more likely the evaluation will have an impact on individuals, teams, and the organization 
overall. However, if there are few means to share what is learned from the evaluation, or organization members are 
restricted from sharing their learning, then evaluation will lose an important opportunity for enhancing the 
organization^ performance. 

The fourth element is the organization^ systems and structures. How employees’jobs are designed, how 
they are rewarded and recognized for their work, and how learning is expected to occur, are all important factors in 
how often and how well evaluation may be conducted by organization members. The more cross-trained they are, 
the more they are encouraged to leam from each another, the more their jobs allow for teamwork, and the more 
employees understand the interrelatedness of their jobs, the more likely evaluation will serve its learning function. 

Finally, the model we propose takes into account the fact that organizations, more than ever, are being 
influenced by a myriad of external forces (Drucker, 1997; Judy & D’Amico, 1997; Marquardt, 1999) (see Figure 1.). 
These include increasing competition for personnel and other resources, evolving customer or client expectations, an 
increasingly diverse workforce, new requirements for working in a global environment, technological advances 
which are literally altering the way we work and communicate, and ever-changing legal rules and regulations. 
While HRD professionals do not need to conduct a complete scan of these variables when designing and 
implementing an evaluation, they should at least be aware of where and how the organization is responding to these 
factors at any point in time. It is entirely possible that an evaluation could be seriously affected by any one of these 
variables. 



The Contribution of the Evaluation Model to HRD Practice and Research 

Many writers on management today believe that the future success of organizations will be dependent on 
their ability to build core competencies within a context of collaboration. Technology, and quick and easy access to 
information will help create web-like structures of work relationships which will facilitate their working on complex 
organizational issues (Hargrove, 1998; Helgeson, 1993; Limerick & Cunnington, 1993; Stewart, 1997). We believe 
evaluation can be a means for a) collectively identifying information needs, b) gathering data on critical questions, 
and c) providing information that when used, becomes part of the organization^ knowledge base for decision- 
making, learning, and action. As organizations have been forced to respond to an increasingly competitive 
environment that is volatile and unpredictable, and as they are likely to continue being pressured to do things better, 
faster, and cheaper, they are looking at evaluation as a means to help them determine how best to proceed. For the 
HRD practitioner, the proposed model offers a kind of roadmap for undertaking an evaluation. Instead of deciding 
to gather only participant reactions, the practitioner can use the model to determine the political context and 
intended uses of findings in order to determine what questions must be answered by the evaluation. By doing so, the 
practitioner will be following a more comprehensive and systematic evaluation process. Such a process should 
increase the probability that the evaluation findings are valid and can be used for decision-making. 

For the HRD researcher, the proposed model offers new research opportunities. For example, HRD 
scholars could examine the specific effects of evaluator characteristics, the political context, and the intended use on 
decisions regarding the evaluation design of a learning and performance initiative. Furthermore, researchers need to 
determine the effects of these factors on actual use of evaluation findings. Other questions might revolve around the 
effects of the evaluation process on organizational decision-making. For example, do certain evaluation designs and 
methods tend to emerge from certain evaluator characteristics, political contexts, or intended uses? What is the 
impact of certain evaluation processes on organizational decision-making? Do organizations or functions within 
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organizations that routinely evaluate programs and processes yield higher levels of learning and performance than 
those organizations or functions that do not evaluate? 

In the knowledge era, where we now find ourselves, it is critical that organizations learn from their 
experiences, that they see themselves as part of a larger system, and that they use quality information for making 
timely decisions. We hope that the proposed evaluation model will provide HRD practitioners and researchers with 
a tool that can help them collect and use valid information that contributes to individual, group, and organizational 
growth and success. 
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Economic Analysis of Human Resource Development: Update on the Theory and Practice 



Richard A. Swanson 
University of Minnesota 



A serious problem facing the HRD profession is the perception among executives that workforce 
competence and expertise are essential while HRD is optional. Studies of HRD practices report that 
almost no HRD functions are assessed in terms of their overall economic contribution to the 
organization. This paper reports economic research related to the contribution HRD makes to the 
success of organizations and proposes new areas of economic research. 



Key words: economic analysis, benefit analysis, financial benefit analysis 

Does business education provide value for the money? This is one of the pressing questions posed in the recent 
business book titled Graw Training (Crainer & Dearlove, 1999). Couple this message with headlines like 
“Training falls down on the job ” ( Daily Telegraph , October 27, 1997, p. 31) and the popular perception that HRD 
costs organizations more than it returns in benefits continues. 

Any organization that remains alive will ultimately judge each of its components from a retum-on- 
investment (ROI) framework and they will do it with or without valid data. Not only will the judgement be made; 
actions will be taken based on the economic assessment (real or perceived economic data). These hard decisions 
are not restricted to private sector business and industry. Every organization is ultimately an economic entity. 
Organizations do not have an inherent right to continue to exist. Two examples that quickly come to my are the 
closing of my childhood church over 20 years ago and a more recent closing of a non-profit ballet performance 
company. 

To face this challenge, four views of HRD have been presented to the HRD profession. They are: (1) a 
major business process, something an organization must do to succeed, (2) a value-added activity, something that 
is potentially worth doing, (3) an optional activity, something that is nice to do, and (4) a waste of business 
resources, something that has costs exceeding the benefits (Swanson, 1995). 

The dominant view of HRD is within the last two options above- HRD as being an optional activity or 
having costs greater than its benefits. The simple idea that HRD is not a good investment is popular and 
entrenched. At the same time HRD professionals most often believe that what they do is a good investment. The 
popular belief within the profession has very little evidence to back it up with only three percent are evaluated for 
financial impact (Bassi, Benson, & Cheney, 1996). 



Problem Statement, Historical Framework, and Research Questions 

Top decision-makers in organizations create scenarios and strategies that provide essential and fundamental 
organizational direction. These decisions are ideally based on estimates of ftiture states and what is required to 
attain them. While HRD theorists and leaders may think of HRD as essential, strategic, and a sound investment, it 
is the perspectives that top decision makers have on knowledge, competence, and expertise that fundamentally 
limits the role of HRD in an organization (Herling & Provo, 2000). 

HRD leaders propose strategies, projects, and programs to top management. Unlike other managers, HRD 
people tend to resist these strategic tasks, especially when they are tied to economic issues. While much claim 
economic theory to be fundamental to the profession (Ruona & Swanson, 1998), HRD people are not inclined 
toward the financial side of the organization. 

Historical Framework 

Economic thinking related to human capacity, human expertise, and human effort and the effects of each 
is disjointed. History provides a fairly consistent notion that there is much to be gained by being purposeful in 
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managing these domains. Throughout history, the ideological responses to capturing the spoils of human expertise 
have ranged from communes, to slavery, to meritocracies. 

The importance of increasing onefc expertise is confirmed in society^ comparisons of educational levels 
and economic success. Even so, investments in the development of its personnel are still not a clear option for most 
firms. Organizations can access expertise in ways other than offering development programs. For example, they 
can hire expertise and/or establish the expectation that employees will manage the development of their own 
expertise. Neither of these two options requires an organization to make direct financial outlays for HRD. 

For the HRD profession, the ‘Training Within Industry” project (Dooley, 1945) was a w atershed. This 
1940-45 massive national performance improvement effort clearly and consistently demonstrated the economic 
impact of HRD and the required conditions for achieving financial benefits. The role of this national effort in 
shaping the contemporary HRD profession (see Ruona & Swanson, 1998) and for directly connecting the 
profession to economic results cannot be underestimated. Unfortunately, the informed and best practices in HRD 
during WWI slowly eroded during the postwar affluence in a manner similar to the quality of USA produced 
goods. 

In the 1970s a renaissance in the profession provided incentive to think more about HRD as an 
investment. The literature increasingly reported financial analysis methods (FAM) and studies of programs’ costs 
and benefits (Cullen, Sawzin, Sisson, & Swanson, 1976, 1978; Gilbert, 1978; Meissner, 1964; Swanson & Sawzin, 
1975). In the 1980s this financial analysis trend continued with a greater focus on costs and the human resource 
management perspective versus performance improvement (Cascio, 1987; Flamholtz, 1985; Head, 1985; Kearsley, 
1982; Spencer, 1986). These company-wide FAMs took an accounting perspective rather than a performance 
improvement perspective. 

To the 1980s, FAM efforts in HRD did not address the decision-making dilemmas faced by organizations 
at the investment decision stage of their organizational planning. Difficult as it may seem, any organization can 
conduct an after-the-fact cost-benefit analysis. What was needed was a method for forecasting those costs and 
benefits, at the point of making investment decisions. The forecasting financial benefits (FFB) of HRD method was 
designed to fill this gap (Swanson, Lewis, & Boyer, 1982; Swanson & Geroy, 1983; Swanson & Gradous, 1988). 
The FFB is a practical step-by-step method for making accurate investment decisions based of forecasting (1) the 
financial value of improved performance projections for a program, (2) the cost of implementing a program, and 
(3) the return on the program investment (Swanson & Gradous, 1988). The FFB method is best suited to short term 
HRD interventions purposefully connected to performance deficiencies. 

Problem Statement and Research Questions 

The problem facing the established FAMs are not easily applied to large-scale change, long-term change 
and to interventions loosely connected to performance requirements. The following two questions serve as the basic 
of this inquiry: 

1. What are the general findings relevant to the financial analysis of HRD have appeared in the literature? 

3. What economic theories and tools reported in the literature should HRD pursue to assess the HRD function and 
large-scale, organization-wide change efforts? 



General Findings Related to HRD Financial Analysis Methods 

The basic financial analysis method (FAM) method, based upon several years of research, has proven to be a 
helpful tool to overcome the difficult and often resisted problem of talking about human resource development in 
dollar and cents terms (Swanson & Gradous, 1988). The model and method for analyzing actual and forecasted 
financial benefits are relatively simple and straightforward. They both have three main components: (1) the 
performance value resulting from the program, (2) the cost of the program, and (3) the benefit resulting from the 
program. The basic financial analysis model is: 

Performance Value 

- Cost 

Benefit 

The FAM method is an expansion of the three components into three separate worksheets. Readers 
wishing to receive detailed instruction on these should obtain the author^ full text (Swanson & Gradous, 1988). 
For a broad overview of FAMs in context of HRD, see Mosier (1990). In addition, the extensive bibliography 
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serves to capture important literature related to the economic analysis of HRD beyond just those studies cited 
directly in this paper. 
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Early HRD Financial Analysis Classics 



There is a substantial base of HRD economic research. Unfortunately, it spread throughout the refereed, 
non-refereed, HRD, and non-HRD literature and generally is not in the hands of HRD decision-makers. Five early 
classics provide excellent examples. The research results from these varied studies were quite consistent. They 
demonstrated that HRD imbedded in a purposeful performance improvement framework yielded very high returns 
on investments, an ROI of 8: 1 or more in a year or less (see Swanson, 1998). 

From Financial Analysis of Methods (FAM) to Forecasting Financial Benefits (FFB) 

There are a substantial number of studies in the realm of forecasting the financial benefits of HRD. This 
FFB literature is also dispersed in the HRD literature. These studies in this domain clearly demonstrate that HRD 
can be a very sound financial investment (see Swanson, 1998). Research further provides evidence that HRD 
interventions focused on appropriate dependent performance variables and systematically executed will financially 
forecast and return 8:1 or more. In contrast, there is no evidence that unfocused and unsystematic HRD 
interventions yield positive returns, let alone returns that exceed the costs. 

Recent Financial Analysis Research in HRD 

There is a substantial array of new HRD related economic research studies. Each of these studies provides 
a challenge and opportunity in the financial analysis of HRD benefits. The most interesting are the Critical 
Outcome Technique (Mattson, 2000) and the efforts at industry-wide impact (Lyau & Pucel, 1995). 



Economic Assessment of the HRD Function and Organization-wide Efforts 

Companies tend to use a limited number of economic analysis techniques. They include: payback time, average 
rate of return, present value (or present worth), and internal rate of return ( Moore & Reichart, 1983). 

Payback time is the time period in which the amount invested is recovered by financial returns. This 
technique typically avoids long-range issues and also doesn't capture any long-range benefits. Average rate of 
return allows for a comparison of alternatives as to their relative return per dollar invested. Alternatives can be 
ranked or compared to a standard. Present value recognizes that money has value over time and could be making 
money in other investments. Thus, cash flows are discounted to the present using a standard interest rate . Internal 
rate of return is a method that determines the interest rate needed to make the present value of the cash 
investment. It represents the rate of interest it would take if all efforts were paid from borrowed funds. 

The economic techniques in the previous discussion force the decision-maker— and HRD decision makers 
in particular- to realize that money to make an organization function is not sitting idly in a safe. If there is money 
available, there are alternative means of increasing its worth. If it is not available, and must be obtained through 
loans or from selling shares to stockholders, the bank and the shareholders expect a return for allowing the use of 
their money. 

Three economic assessment strategies are highlighted here to explore the challenge of getting beyond the 
perspective of individual program assessment. They each deserve attention and application in HRD. They are: 
•Measuring Intellectual Capital 
•Measuring Human Capital 
•Strategic Training Investment Decision Model 

Measuring Intellectual Capital (Edvinsson & Malone, 1997) 

Premise: In a knowledge-based company, the accounting system doesn't capture anything important. 

Value lies in assets and when the assets are intangible, accounting has great difficulty. 

Method: A measurement system for company-wide intellectual capital (IC); 3 1 possible IC metrics (e.g., 
information technology investment in dollars, rate of repeat customers in percentage) that can be 
customized, pursed, and tracked in terms of gains in the metrics and the overall economic performance of 
the company. 
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Measuring Human Capital (Provo, 2000) 

Premise: Like other assets, there is a return on human assets. Human capital is a source of strategic 
advantage and requires investment. Build on the value chain, searches for constraints, and uses data 
models even if the data are imperfect. 

Method : A "return on people approach" (Boudreau & Ramstad, 1997). The formula is: Effectiveness 
(increase in capabilities) x Impact (value of capabilities) = Benefit (value created). 



Strategic Training Investment Decision Model (Krone, 2000). 

Premise: There is strategic economic risk associated with investing in a training initiative. There is (1) 
general training that raises the productivity the same for the firm providing the training as in other firms 
and (2) specific training provided by a firm has value to that firm and no utility to competitive firms 
(Becker, 1993). 

Method: The training investment decision is based on the expected return on specific training, plus the 
expected return from general training, minus the performance improvement value of the training to the 
competing organizations 



Conclusion 

Economics has been purported to be one of the foundational theories of the HRD discipline. Yet, the HRD 
profession is woefully behind in reporting its economic contribution. This paper dichotomizes economic analysis 
into: (1) the individual program level and (2) the larger function or multi-dimensional organization-wide 
intervention level. The financial analysis methods available to the HRD profession are more than adequate to do 
the economic analysis of individual programs. In contrast, the methods and reported research economic analysis of 
the total HRD function or multi-dimensional organization-wide interventions is inadequate. Economic research 
into the underlying economic theory and application to HRD is needed to fill this void. 
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Evaluating HRD Research Using a Feminist Research Framework 
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Over 400 AHRD Proceedings papers from 1997-1999 were analyzed according to a feminist 
research framework to understand how HRD creates knowledge through research. Although 
knowledge is being produced through both traditional and non-traditional methodologies , few 
studies recognize gender as a category of analysis. Nearly absent from the literature are studies 
concerned with: women/diverse peopled experience ; asymmetrical power arrangements; 
problems of racism and sexism; or advocacy of social justice and change. Research implications 
research are discussed. 



Keywords: Research, Feminism, Document Analysis 

Human Resource Development is an emerging discipline that is in the process of creating and validating knowledge. 
Little has been written about HRD research itself (Hixon and McClemon, 1999; Jacobs, 1990; Sleezer and Sleezer, 
1997, 1998). Existing studies of HRD research have investigated journals publishing HRD research (Hixon and 
McClemon, 1999; Sleezer and Sleezer, 1997, 1998), and analyzed study context and methods (Hixon and 
McClemon, 1999). Arnold (1996) examined the 1994 and 1995 proceedings of AHRD according to the type of 
research and tools used to discuss findings, and van Hooff and Mulder (1997) described the contents and 
characteristics of research appearing in the 1996 AHRD proceedings, van Hooff and Mulder found that most 
researchers conduct studies of individual development and on the development of HRD as a field. Key research 
issues include: integrity, globalization, teams, employee development, learning on-the-job, new technologies, 
transfer, evaluation, organizational change, training effectiveness, partnership research, and roles in HRD. HRD has 
an impressive and diverse body of research. As the field emerges it is important to not only continue existing 
research, but also critically assess what is and is not being studied. 

HRD as a discipline, has not exceedingly concerned itself with issues of diversity, equality, power, 
discrimination, sexism, racism, or other similar issues in organizations. Yet, these challenges pervade in both the 
workplace and society. Governmental policies, business practices, and research agendas lag behind the pace of 
workplace diversification. Fernandez observes, ‘fcorporate America as a whole, has failed to effectively address the 
challenges posed by diversity, particularly with regard to racism and sexism” (1999, p. 3). Meyerson and Fletcher 
(2000) suggest that discrimination in organizations is so deeply embedded culturally that it is practically 
indiscernible. Whether you consider race or gender, the figures are grim. Beck reports that 99 percent of all 
American women will work for pay at some point in their lives (1998). Although womenfc workforce participation 
has steadily increased and shows no sign of diminishing in the new millennium, women trail men in pay, promotion, 
benefits, and other economic rewards (Knoke & Ishio, 1998). Despite the progress over the last fifty years, about 
half of the worldfc workers are in sex-stereotyped occupations, and women work in a narrower range of occupations 
than do men. Rowney and Cahoon (1990) suggest that women find it easier to obtain leadership positions at lower 
levels in the hierarchy. According to their figures women hold only 23 percent of managerial positions in Canada, 
which is similar to the United States (24 percent) and the United Kingdom (19 percent). They found, in a sample of 
423 organizations, that 30 percent of first line supervisors were female, whereas only 17 percent of middle managers 
and 8 percent of executives were women (1990). Despite more equal opportunity, women are still segregated into 
typically ‘female” careers, and the wage gap persists. Women earn 76 cents for every dollar men earn (Bureau of 
Labor Statistics, 1998), with the average managerial level differential at 74 cents. The data worsen based on race. 
African-American women earn 58 cents; Hispanic women earn 48 cents, Asian/other women earn 67 cents 
(Catalyst, 1997). 

Women at the top levels are still a rarity comprising only 10 percent of senior managers in Fortune 500 
companies. Fewer than 4 percent of women hold positions of CEO, president, executive vice president and COO, 
and less than 3 percent of top corporate earners are women (Meyerson and Fletcher, 2000). Carly Fiorina became 
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the first woman CEO of a Fortune 100 company when she was named CEO of Hewlett-Packard during 1999. She is 
one of only three women CEOfc in the Fortune 500. Dobosz notes, ‘Fiomia, briefly forgetting her math, cavalierly 
told the press that her appointment proved that there was no glass ceiling” (2000, p. 21). The glass ceiling is alive 
and well although Meyerson and Fletcher suggest it has gone ‘Underground.” They explain, ‘Today discrimination 
against women lingers in a plethora of work practices and cultural norms that only appear unbiased. They are 
common and mundane — and woven into the fabric of an organization^ status quo — which is why most people dont 
notice them, let alone question them. But they create a subtle pattern of disadvantage, which blocks all but a few 
women from career advancement” (200, p. 128). Hultin and Szulkin studied Swedish workers to investigate gender 
wage inequality, specifically whether earnings were affected by the gender composition of the establishments’ 
managerial and supervisory staff. They found that ‘Render-differentiated access to organizational power structures 
is essential in explaining womenfc relatively low wages” (1999, p. 453). They emphasize that gender composition 
in hierarchical power structures should be considered an important part of research to increase our understanding of 
gender wage inequity. They conclude that, ‘Power relations in work organizations are of crucial importance for 
understanding how gender inequalities in financial rewards are generated and sustained in the labor market” (1990, 
p. 465). 

Workplace HRD is not exempt from this type of systemic discrimination. Knoke and Ishio (1998) 
conducted longitudinal data analysis on a cohort of young workers to document that womenfc participation in 
company training programs was at a significantly lower rate than menk Their study was done to evaluate whether 
reports of a demise of the gender gap in company training based on incident levels observed in cross sectional 
surveys were accurate. Knoke and Ishio indicated that, “Our principle conclusion is that the gender gap in company 
job training remains far more robust, tenacious, and resistant to explanation that previous researchers had indicated. 
This discovery admonishes both firms and social researchers to pay more attention to the ways that employees’ 
genders interact with private-sector policies and practices” (1998, p. 153). This study was undertaken heeding pleas 
for more gender sensitive research (Hultin & Szulkin, 1999; Knoke & Isho, 1998) and recognizing Meyerson and 
Fletcher^ (2000) argument that discrimination is embedded so deeply that we do not often see it. 



Problem Statement 

HRD is an emerging discipline. Now is the time to question the theoretical frameworks and practices defining the 
field before they become embedded and simply serve to reinforce the status quo. Todayfc HRD research and 
practice pays significant attention to the US corporate context, skews loyalties toward management, and lauds 
performance improvement above other results. HRD researchers must explore the assumptions underlying their 
research, consider the beneficiaries of research, reflect on areas yet unexplored, and question the value of HRD 
research according to its impact on theory, practice, organizations, communities, and employees. 

Research questions in the social sciences have traditionally been conceptualized without consideration of 
women (Fine, 1985; Lykes & Stewart, 1986; Unger, 1983) and HRD is no exception. A quick reading of HRD 
research reveals an agenda driven by management interests focused primarily on learning and performance. 
Leimbach and Baldwin (1997) identify the characteristics of effective HRD research as being customer driven, 
linked to value creation, short in duration, and rigorous. These characteristics are important in HRD research, 
however, they overlook issues related to women and other diverse individuals, power relationships, social context, 
or social and political change. Employees are not even mentioned in the characteristics. 

The purpose of this inquiry is to understand how the HRD field is creating knowledge through research. 
We are intrigued by what is valued knowledge in HRD and how the research trends over time shape the emerging 
field of HRD. The questions guiding this research are, ‘To what degree is HRD research using feminist inquiry? 
What are some under-addressed dimensions for the emerging field of HRD?” 



Theoretical Framework 

This study is grounded in a feminist research framework. Describing the many types of feminism is beyond the 
scope of this paper. A feminist— at the most simplistic level— is a person who seeks economic, social and political 
equality between the sexes. Feminists participate in and/or support organized activity to advance womenk rights 
and interests. Acker, Barry, and Esseveld (1983) define feminists as engaged in: acknowledging the exploitation, 
devaluation and often oppression of women; making a commitment to changing the condition of women; and 
adopting a critical perspective toward dominant intellectual traditions that have ignored and /or justified womenfc 
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oppression. Pritchard (1994) suggests that ‘Feminist critique starts with Women’ or Womenfc issues’ but goes 
beyond to the impact of gender relations and gendered conditions of human development in all spheres of thought 
and action” (p. 42). Conduit and Hutchinson (1997) summarize five strategies viewed by feminist researchers as 
needed to create change and measure progress. These include: (1) providing women access and encouragement to 
enter the world of work, politics, and knowledge production; (2) reclaiming the works of previously absented 
women (or absent treatments of women) to what is considered the fundamental canon of knowledge in a given 
discipline; (3) adding women into the pot of ‘the human subject” in all types of research where they have been 
previously represented only through their absence; (4) turning feminist eyes, of diverse types, to a reexamination of 
the fundamental theories, mechanisms of analysis, and primary values that have given shape to our epistemological 
techniques and our ontological assumptions; and (5) ascertaining how adding women and feminism to the mix at so 
many levels has changed the kinds of questions asked, the types of policies attended to, and the categories of 
research done. 

Feminist critique examines power in social and political institutions and the values and communication 
patterns that manifest themselves in both abstract and concrete patterns. Feminist critique has revolutionized 
workplace analysis and the ways knowledge is constructed (Pritchard, 1994). See Bierema (1998a; 1998b) for 
further explanation about feminist research in HRD. 



Methodology 

We conducted document analysis of AHRD conference proceedings from 1997-1999 using a feminist research 
framework. We selected a feminist framework because we felt it provided the most critical lens for assessing HRD 
research. Feminist research is concerned with eliminating all forms of oppression, thus it was suitable for 
considering research on diverse populations. Worell (1996) credits feminist psychologists with introducing a 
dialogue that challenged prevailing structures of knowledge creation. Feminist research is grounded in assumptions 
that, like most other social institutions, the process of knowledge creation and dissemination has historically been 
the province of white men. Womenfc experience and knowledge has been traditionally excluded or overlooked in 
social science research. During the last two decades, feminist social scientists have critiqued the research process 
and provided definitions of feminist research. 

Bierema (1998a; 1998b) conducted an exhaustive review of the literature on feminist research and a 
preliminary analysis of 1997 AHRD proceedings. Based on the literature review, Worell and Etaughfc 1994 
framework defining feminist inquiry was selected as the model to evaluate AHRD research. Worell and Etaugh 
(1994), synthesized feminist theory and research in psychology and other disciplines to establish “Themes and 
Sample Variations in Feminist Research.” Worell adapted the list and suggests that feminist research: 

1 . Challenges traditional scientific inquiry. 

2. Focuses on the experiences and lives of women. 

3. Considers asymmetrical power arrangements. 

4. Recognizes gender as an essential category of analysis. 

5. Attends to language and the power to ‘hame.” 

6. Promotes social activism and societal change (1996, p. 476). 

We carefully studied this framework and modified it for the present study to make it appropriate for assessing HRD 
research. Refer to Appendix 1 for a description of the six categories. 

We selected AHRD proceedings for analysis rather than the HRDQ, as we believe the proceedings provide 
a broader view of all HRD research, not only research that is published in journals. We intend to analyze the first 
ten volumes of HRDQ and complete analysis of AHRD proceedings prior to 1997 in the future. 

Data Analysis 

Four hundred and eight proceedings papers were critically reviewed for this study. After reading the 
articles, research methodology was recorded and each paper was assessed to evaluate whether or not it met any of 
the six categories. All papers were considered. Some papers such as forum debates or articles on how to publish in 
journals were not considered in the analysis, and were coded ‘bther.” After eliminating these papers, 396 were 
analyzed according to our framework. Data were recorded on a spreadsheet. Papers qualified for each of the six 
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categories if they met one or more of the points specified by the Feminist Framework for Evaluating HRD Research 
(Appendix 1). 



Findings 

A wide variety of research methodology is applied in HRD. Studies were classified according to traditional 
(experimental, quantitative), non-traditional (multiple methods, qualitative), conceptual/theoretical, review of the 
literature, or other (forums, journals). Table 1 shows the breakdown by type of methodology applied. There is a 
balance of both quantitative and qualitative research in HRD. There were many conceptual/theoretical papers as 
well. This corresponds with Hardyfc (1999) meta-study of HRD research. He suggests that emerging fields engage 
in a considerable amount of theory generation and qualitative research to form a theoretical foundation, and 
gradually shift toward quantitative studies to validate theory. 



Table 1 



A HRD Conference Proceedings Methodology 



Methodology 


1997 

# 


% 


1998 

# 


% 


1999 

# 


% 


1997- 

# 


1999 

% 


Traditional/Experimental 


40 


32.0 


43 


34.68 


61 


38.36 


144 


35.29 


Non-traditional 


45 


36.0 


40 


32.26 


68 


42.77 


153 


37.50 


Conceptual/Theoretical 


25 


20.0 


24 


19.35 


26 


16.35 


75 


18.38 


Review of literature 


12 


9.6 


10 


8.06 


2 


1.26 


24 


5.89 


Other 


3 


2.4 


7 


5.65 


2 


1.26 


12 


2.94 


Total 


125 


100 


124 


100 


159 


100 


408 


100 



Table 2 



1997 AHRD Conference Proceedings according to Feminist Framework for Evaluating HRD Research 





1997 


1998 


1999 


1997- 


■1999 




n= 


122 


n= 


117 


n= 


157 


n= 


396 


Feminist Framework 


# 


% 


# 


% 


# 


% 


# 


% 


1. Challenges traditional scientific inquiry. 


50 


40.98 


45 


38.46 


71 


45.22 


166 


41.92 


2. Focuses on women's experiences and lives 


6 


4.92 


3 


2.56 


11 


7.01 


20 


5.05 


3. Considers asymmetrical power arrangements. 


7 


5.74 


6 


5.13 


7 


4.46 


20 


5.05 


4. Recognizes gender/diversity as category of analysis. 


11 


9.02 


11 


9.40 


20 


12.74 


42 


10.61 


5. Attends to language and the power to name. 


7 


5.74 


6 


5.13 


7 


4.46 


20 


5.05 


6. Advocates social activism and change. 


12 


9.84 


7 


5.98 


9 


5.73 


28 


7.07 



Table 2 shows the papers according the Feminist Framework for Evaluating HRD Research. It examines how many 
papers met the six categories. Each of the categories was assessed against the total manuscripts by year and 
cumulatively for 1997-1999. Challenging traditional scientific inquiry is the most frequent category with nearly 38 
percent of the papers falling into this category over three years. The next most frequent category is recognizing 
gender as a category of analysis. Studies met this criterion over 10 percent of the time over three years. Far less 
frequent were the categories: 2) focuses on the experiences of women/diverse groups; 3) considers asymmetrical 
power arrangements: 5) attends to language and the power to name; and 6) advocates social activism and change. 
These occurred in 5-7 percent of the papers from 1997-1999. 

Table 3 summarizes the number of studies according to the number of categories that were present within a 
single study. Unfortunately there were few that met more than one category (usually the research category). Over 
half of the articles failed to meet the categories whatsoever. 

Table 3 



Number of Categories met 1997-1999 





1997 

n=122 


1998 

n=117 


1999 

n=157 


1997- 1999 
n=396 


Methodology 


# 


% 


# 


% 


# 


% 


# 


% 


Meets all 6 categories 


0 


0 


1 


.855 


1 


.64 


2 


.51 


Meets 5 categories 


3 


2.46 


0 


0 


3 


1.91 


6 


1.52 


Meets 4 categories 


2 


1.64 


3 


2.56 


2 


1.27 


7 


1.77 


Meets 3 categories 


5 


4.10 


3 


2.56 


6 


3.82 


14 


3.54 


Meets 2 categories 


7 


5.73 


4 


3.42 


8 


5.10 


19 


4.79 


Meets 1 category 


41 


33.61 


43 


36.80 


64 


40.76 


148 


37.37 


Meets none of the categories 


64 


52.46 


63 


53.85 


73 


46.50 


200 


50.50 


Total 


122 


100 


117 


100 


157 


100 


396 


100 



Discussion 

This section revisits the Feminist Framework for Evaluating HRD Research and assesses the findings against it. It 
also makes some recommendations for future HRD research. Although these criteria focus on research models that 
challenge scientific inquiry, experimental designs are considered feminist research if they meet the other criteria set 
forth in Appendix 1, and they were considered for this analysis. DeVault (1996) implores researchers to avoid 
favoring one philosophical research stance over the other and suggests that ‘feminists have made major 
contributions by finding concepts and practices that resist dualisms’ and they urge resistance to the qualitative- 
quantitative division” (p. 31). Quoting Cannon, Higginbotham, and Laung, she points out that small scale projects 
may be more likely than quantitative studies to reproduce race and class biases of the discipline by including 
participants who are readily accessible to the researchers. She also advocates that ‘Quantitative feminist work 
involves correcting gender and other cultural biases in standard procedure” (p. 31). Quantitative methods are 
customarily used in HRD research and researchers would improve knowledge about race and gender by heeding 
DeVaultfc advice. Qualitative research is also widely applied in HRD research, however, very few authors 
acknowledge their role as the researcher or are forthcoming about their biases. We believe all types of research have 
value and encourage research that strives to eliminate bias, promote equity, reveal corrupt or harmful practices, and 
improve the standing of all employees. 

Five percent of the papers over three years dealt with womenfc issues and even fewer addressed diversity. 
Articles on individuals possessing a double minority status, such as Black women, were almost non existent. We 
found no articles addressing how women have contributed to HRD history or that examine the multiple 
representations of women in society (they are not all white, middle class). We conclude that the questions relevant 
to working women and diverse groups, such as harassment, discrimination or job segregation, are not being 
adequately addressed in HRD research. We agree that HRD research must aggressively pursue these topics in a 
diversifying, globalizing workplace. Understanding how power is wielded in HRD is not a priority according to our 
analysis finding that only 5 percent of the papers addressed asymmetrical power arrangements. We found this 
particularly stunning considering the impact that group and power dynamics has on all organizational activities. We 
are encouraged that over 10 percent of the papers from 1997-1999 recognized gender as a category of analysis. 
What was surprising, however, was that many more studies report women in the sample, but fail to include analysis 
based on gender. All participants are lumped into the same category and measured. HRD research must move 
beyond counting women and diverse groups in the sample to analyzing the data according to these categories. 

Although HRD discusses ‘bndiscussibles” in the context of the learning organization, the field has not 
progressed to addressing undiscussibles related to gender and diversity in research. Five percent of the studies 
analyzed addressed language and naming. Perhaps we are caught in the fray of political correctness where the overt 
discussion of the dynamics and impact of racism and sexism cannot be stomached. This silence only contributes to 
a discourse that margninalizes women and diverse beings in organizational context. HRD needs to delve into the 
‘isms” that are not readily spoken about. HRD needs to address workplace discourse and how it silences, teaches, 
and oppresses humans. 

Although HRD is a field committed to organizational change, there is little advocacy of change beyond the 
walls of the corporation. Our findings are telling in that only 7 percent of the papers analyzed advocated social 
activism and change. These findings indicate that HRD lacks a social conscience where research is concerned. We 
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realize that the space limitations of the proceedings papers may have inhibited thorough discussion of social issues, 
but we were startled at the established pattern of overlooking it in study conceptualization, analysis, and discussion. 



Limitations 

We acknowledge that this analysis is imperfect. This study was not a critique of the theories and practices of HRD, 
but rather an analysis of HRD research. We recognize that research cannot be free from culture, history or 
experience. We are committed to diversity and equity in both the workplace and HRD field. Both of us are white, 
able-bodied, heterosexual women. One of us is European and the other North American. Through our work in many 
different organizations and industries, we have different experiences with gender-related issues. We have trained in 
both HRD and adult education. The analysis of the studies was an inductive, subjective process that was influenced 
by our experience. To ensure consistency we had multiple discussions about how studies would qualify for each 
category and discussed studies that were questionable regarding their fit with the framework. We also talked at 
length about what would exclude a study. We shared long-winded dialogue about what should be considered 
advocating social change and about how much we could infer in the analysis. We are aware that organizational 
development and change has the potential to create workplace equity, however, we took a conservative approach to 
attributing the advocacy of social justice and change as a criteria. We considered only articles that made a strong 
statement advocating change in the organization that was committed to addressing issues of diversity or inequity. 
We also assessed articles that had the potential to have a social impact. Above all, we decided that social justice and 
change could not be inferred in our analysis, but rather had to be overtly stated by the author(s). Our work is not 
finished. We have yet to conduct a cappa statistic to validate our interrater reliability and have future plans to do 
this. We also intend to continue analyzing HRD research according to this framework. Finally, our bias is that the 
HRD field would benefit from research that is more critical and inclusive methodologically, and should strive to 
create knowledge that promotes social change and workplace equity. 



Implications 

This analysis of over 400 AHRD papers, representing the most comprehensive, current HRD research to date, 
discloses what HRD is not focusing on: issues of equity and access in the workplace. Although we are encouraged 
that HRD has some highly dedicated researchers who are creating knowledge that addresses the issues raised in the 
feminist research framework, we are also disheartened that the field is not doing more toward creating a diverse, 
empowered workplace. Other than promoting alternative research designs, and sometimes using women (or 
diversity) as a unit of analysis, HRD focuses little on issues of social justice in the workplace or larger social 
context. Womenfc experience is ignored, as are asymmetrical power arrangements. Gender is not used as a 
category of analysis — even when data are collected by gender. Organizational ‘Undiscussables” such as sexism, 
racism, patriarchy, or violence receive little attention in the literature, yet have considerable impact on 
organizational dynamics. Finally, HRD research has only weakly advocated change. These findings are cause for 
alarm. Is HRD research reproducing existing power relationships in organizations? Is HRD research in the service 
of corporate executives and shareholders? What are responsible HRD researchers to do? We suggest a modest 
starting point of reviewing the Feminist Framework for Evaluating HRD Research and assessing how planned, 
ongoing, and completed studies address the issues it highlights. 

Although there is a dearth of feminist research in HRD, this trend is not unique to HRD. Conduit and 
Hutchinson (1997) examined eight major public administration and policy journals to assess how many women were 
publishing, their patterns of publication, and content of the research. They found that women are historically 
underrepresented in most, but more recently have been publishing at rates commensurate with their representation in 
the academy. They found a surprisingly small number of articles by women and men scholars that addressed 
traditional womenfc policy concerns or applied a feminist perspective to the research. They conclude that public 
administration has yet to be significantly influenced by the research and theoretical contributions of the womenfc 
movement found in the greater academic community. We concur with Condit and Hutchinson^ (1997) conclusion, 
that” when it comes to the question of the emperorfc new clothes, the emperor still seems quite bare , ’(p. 194). 
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Appendix 1 

Feminist Framework for Evaluating HRD Research * 



1. Challenges traditional scientific inquiry 

♦ Rejects the assumption of truly objective science free from culture, 
history and experience of the observer. 

♦ Restructures the polarity of objective-subjective. “Subjects” should 
not become objects to be manipulated by the researcher but 
collaborators in the process. 

♦ Identifies and corrects multiple elements of sexism and bias in 
scientific research procedures. 

♦ Affirms that raw data never speak for themselves and that all data 
require categorization and interpretation. 

♦ Emphasizes the researcher as an individual who interacts with 
participants in meaningful ways that enrich both the observer and 
observed. 

♦ Produces a more inclusive science that reflects alternative realities; 
including multiple perspectives by both researcher and participants; 
expanding the diversity of all person k involved; recognizing that 
reality is created, in part by the scientific process. 

♦ Extends the populations studied beyond White middle class, college 
student samples; studying populations that are relevant to the 
questions being asked. 

♦ Values a range of research methods as legitimate; asserting that 
qualitative, quantitative, ethnographic, and other methods of 
gathering data may be useful for different purposes and may reveal 
unique information. 


2. Focuses on the experiences of women/diverse groups. 

♦ Challenges the category of “women” by exploring diversity 

♦ Affirms womenk strengths, resilience, competence 

♦ Discovers womenk contributions to HRD research/history. 

♦ Values women as a legitimate target of study. 

♦ Studies women apart from the standard of male as norm. 

♦ Questions the category of woman as representative of all women. 
Recognizes/explores sources of variation, e.g., ethnic and racial 
identities, socioeconomic status, sexual orientation, disability, age, 
parenthood, employment status. 

♦ Encourages research questions grounded in personal experiences of 
women researchers. 

♦ Encourages research questions relevant to womenk lives: sexual 
harassment, reproductive processes, employment segregation, 
discrimination, work/life balance, etc. 

♦ Rejects sex difference research as basic to our understanding of 
women, or of men; recognizes differences among women, and 
among men, are greater than those between them. 

♦ Constructs methods of research targeting issues of importance to 
womenk lives such as hostility to women, traditional gender role 
beliefs — methods that may illuminate other observer relationships 
and bias against women. 

♦ Studies women in the context of their lives and natural milieu; 
avoids “context-stripping” through laboratory approaches that 
reduce complexity and individuality. 

♦ Attends to womenk strengths and capabilities as well as their 
problems; researches womenk competency and resilience. 

♦ Views observed gender differences in the context of power 
dynamics and womenk expected socialized role behaviors rather 
than as differences embedded in biology. 


3. Considers asymmetrical power relationships 

♦ Seeks empowerment of all girls and women. 

♦ Recognizes womenk subordinate status in society as based on 
unequal power distribution instead of on deficiencies; explores 
power k influence on the quality of women k lives. 

♦ Considers differences among women as mediated by power 
differentials related to opportunities available based on color, 
economic sufficiency, age, sexual orientation, etc. 

♦ Attends to privilege and privation as sources of questions. 

♦ Examines women k health concerns within the context of power 
arrangements: i.e., maternity leave issues. 

♦ Studies interpersonal relationship within the context of patriarchal 
power arrangements. 

♦ Explores the basis of stereotyped female characteristics, such as 
sociability, nurturance, or passivity in the context of unequal power 
relations; pointing out that what appears natural may be framed by 
the politics of power. 

♦ Shifts attributions of responsibility from victim to perpetrator. 

♦ Seeks strategies leading to women k empowerment. 


4. Recognizes gender (and/or diversity) as an essential category of 

analysis 

♦ Points out the multiple conceptions of gender and diversity; 
challenges the use of gender only as an independent variable that 
explains observed behavior. 

♦ Explores the functions of gender as a stimulus variable that frames 
expectations, evaluations, and response patterns. 

♦ Recognizes gender as a social construction based on power 
arrangements; views observations attributed to gender in the context 
of power asymmetries. 

♦ Emphasizes the situational context of gender and gendering as an 
active process that structures social interactions. 


5. Attends to language and the power to “name” 

♦ Creates public awareness of hidden phenomena by identifying and 
naming them, as in sexual harassment, initiates research on hidden 
phenomena based on the process of naming. 

♦ Restructures language to be inclusive of women; rejects the generic 
masculine; promotes a gender free language system. 

♦ Renames and restructures research topics 

♦ Reduces polarity between private and public in womenk lives, such 
as renaming womenk work, concepts of family, and the appropriate 
placement of these within private and public domains. 

♦ Recognizes that language frames thought and vice versa; attention 
to syntax as power-driven, such as reversing the obligatory ordering 
of male/female, boy/girl, men/women. 


6. Promotes social activism and social change 

♦ Reconceptualizes theories, methods, and goals to encompass 
possibilities for social change, toward reductions in power 
asymmetries and promotion of gender justice. 

♦ Creates a science that will benefit rather than oppress women, and 
that will correct as well as document the prevalence of inequity, 
illness, violence, etc. 

♦ Remains cognizant of how research results may be used, and 
promotes responsible applications of research findings. 

♦ Directs personal involvement and action to initiation or support 
changes in policies, practices, and institutional structures that will 
benefit women and correct injustices. 



♦Adapted from Worell and Etaugh (1994) and Worell (1996) 
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